Image Processing Apparatus, Recording Medium Storing Image Processing Program, And Method Of Image Processing

ABSTRACT

An image processing apparatus including a first image processing unit and a second image processing unit. The first image processing unit is configured to calculate a parameter for image processing based on a first image, but not based on a second image, and execute the image processing on the first image using the parameter. The second image processing unit is configured to execute the image processing on the second image using the parameter.

FIELD OF THE INVENTION

The present invention relates to an image processing apparatus, arecording medium storing an image processing program, and a method ofimage processing.

BACKGROUND OF THE INVENTION

Conventionally, there are cases where it is desired to execute the sameimage processing on a plurality of images. For example, in order tocorrect images contained in a moving image for shaking, JP 2010-10798Adiscloses a technology that calculates parameters for image processingat respective times of day from images at the respective times of dayand corrects the images at the respective times of day using theparameters for image processing at the respective times of day.

However, according to JP 2010-10798A, with respect to a plurality ofimages on which the same image processing is to be executed, parametersfor that image processing are calculated separately. It should be notedthat “the same image processing” as used herein means image processingthat is realized by the same algorithm and includes cases wheredifferent parameters are used. In that case, there is a possibility thatan enormous amount of computation time is taken to execute the imageprocessing on the plurality of images. This problem is especiallynoticeable when image processing parameter calculation is a high-loadprocess.

It is an object of the present invention to provide an image processingapparatus, a recording medium storing an image processing program, and amethod of image processing that can reduce computation time taken toexecute the same image processing on a plurality of images.

SUMMARY OF THE INVENTION

An image processing apparatus according to a first aspect includes afirst image processing unit and a second image processing unit. Thefirst image processing unit is configured to calculate a parameter forimage processing based on a first image, but not based on a secondimage, and execute the image processing on the first image using theparameter. The second image processing unit is configured to execute theimage processing on the second image using the parameter.

Here, the same image processing is executed on the first image and thesecond image. In addition, the parameter for use in the image processingof both of the images is calculated from only the first image of thefirst and second images. In other words, the parameter calculated fromthe first image for use in the image processing of the first image isalso used for the image processing of the second image. It should benoted that as the manner of making use of the parameter calculated fromthe first image for the second image, a manner in which the parametercalculated from the first image is used as it is for the imageprocessing of the second image, a manner in which the parametercalculated from the first image is fine-tuned before being used for theimage processing of the second image, and the like are conceivable. Thatis to say, processing for calculating a parameter for use in the imageprocessing of the second image is omitted or simplified. Accordingly,computation time taken to execute the same image processing on aplurality of images is reduced.

An image processing apparatus according to a second aspect is the imageprocessing apparatus according to the first aspect, wherein the firstimage and the second image are images on the same timeline.

In many cases, it may be appropriate to execute the same imageprocessing using the same parameter on a plurality of images that is atleast a part of images belonging to the same timeline. This is becausethose images are highly likely to have been taken under similarconditions. Here, it is possible to execute the same image processingusing the same parameter on the first image and the second image thatare on the same timeline. Accordingly, it is possible to obtainappropriate images while increasing the speed of image processing.

An image processing apparatus according to a third aspect is the imageprocessing apparatus according to the first aspect, wherein the firstimage and the second image are images on different timelines.

There are cases where it is appropriate to execute the same imageprocessing using the same parameter on a plurality of images belongingto different timelines. For example, this holds in the case where thoseimages have been taken by the same camera installed at the samelocation. Here, it is possible to execute the same image processingusing the same parameter on the first image and the second image thatare on different timelines. Accordingly, it is possible to obtainappropriate images while increasing the speed of image processing.

Image processing apparatuses according to fourth through sixth aspectsare, respectively, the image processing apparatuses according to thefirst through third aspects, wherein the parameter includes an extent ofblurring. The image processing includes processing for enhancing anedge.

Here, the processing for enhancing edges in not only the first image butalso the second image is executed using the extent of blurring of thefirst image that has been estimated from the first image. The extent ofblurring is expressed as, for example, a point spread function (PSF).The extent of blurring is a parameter that depends on the properties oflenses of a camera when it is resulted from in defocusing. Therefore, ifthe first and the second images are images taken by the same camera, theextent of blurring estimated from the first image can be applied to thesecond image. Here, it is possible to obtain appropriate images whileincreasing the speed of edge enhancement processing of a plurality ofimages that have been taken under similar conditions, when the samecamera was used to take those images or the like.

Image processing apparatuses according to seventh through ninth aspectsare, respectively, the image processing apparatuses according to thefirst through third aspects, wherein the parameter includes at least oneof noise information, distortion aberration information, and peripheralillumination information. The image processing includes processing forremoving noise, processing for adding noise, processing for correctingfor distortion aberration, and processing for correcting for peripheralillumination.

Here, at least one of the processing for removing noise, processing foradding noise, processing for correcting for distortion aberration, andprocessing for correcting for peripheral illumination of not only thefirst image but also the second image is executed using at least one ofthe noise information, distortion aberration information, and peripheralillumination information on the first image that has been estimated fromthe first image. The noise information, distortion aberrationinformation, and peripheral illumination information are parameters thatdepend on shooting conditions such as the camera, shooting time of day,and shooting location. Accordingly, it is possible to obtain appropriateimages while increasing the speed of the noise removal processing, thedistortion aberration correction processing, and the peripheralillumination correction processing of a plurality of images taken undersimilar conditions.

Image processing apparatuses according to tenth through fourteenthaspects are, respectively, the image processing apparatuses according tothe first, second, third, fourth and seventh aspects, and furtherincludes an instruction receiving unit. The instruction receiving unitis configured to receive an instruction to execute the image processingon the first image and the second image. The first image processing unitis configured to calculate the parameter and execute the imageprocessing on the first image in response to the instruction. The secondimage processing unit is configured to execute the image processing onthe second image when the parameter has been calculated.

Here, when the user gives an instruction to execute the imageprocessing, the parameter for the image processing is automaticallycalculated, and the image processing is automatically executed on thefirst and the second images. Accordingly, the waiting time for the userfrom when the user gives the instruction to execute the image processinguntil when the image processing is completed is reduced.

A non-transitory computer-readable recording medium according to afifteenth aspect is a recording medium storing an image processingprogram configured to cause a computer to execute a first imageprocessing step and a second image processing step. The first imageprocessing step is a step of calculating a parameter for imageprocessing based on a first image, but not based on a second image, andexecuting the image processing on the first image using the parameter.The second image processing step is a step of executing the imageprocessing on the second image using the parameter.

Here, the same image processing is executed on the first image and thesecond image. In addition, the parameter for use in the image processingof both of the images is calculated from only the first image of thefirst and second images. In other words, the parameter calculated fromthe first image for use in the image processing of the first image isalso used for the image processing of the second image. It should benoted that as the manner of making use of the parameter calculated fromthe first image for the second image, a manner in which the parametercalculated from the first image is used as it is for the imageprocessing of the second image, a manner in which the parametercalculated from the first image is fine-tuned before being used for theimage processing of the second image, and the like are conceivable. Thatis to say, processing for calculating a parameter for use in the imageprocessing of the second image is omitted or simplified. Accordingly,computation time taken to execute the same image processing on aplurality of images is reduced.

A non-transitory computer-readable recording medium according to asixteenth aspect is the recording medium according to the fifteenthaspect, wherein the parameter includes an extent of blurring. The imageprocessing includes processing for enhancing an edge.

Here, the processing for enhancing edges in not only the first image butalso the second image is executed using the extent of blurring of thefirst image that has been estimated from the first image. The extent ofblurring is expressed as, for example, a point spread function (PSF).The extent of blurring is a parameter that depends on the properties oflenses of a camera when it is resulted from in defocusing. Therefore, ifthe first and the second images are images taken by the same camera, theextent of blurring estimated from the first image can be applied to thesecond image. Here, it is possible to obtain appropriate images whileincreasing the speed of edge enhancement processing of a plurality ofimages that have been taken under similar conditions, when the samecamera was used to take those images or the like.

A non-transitory computer-readable recording medium according to aseventeenth aspect is the recording medium according to the fifteenthaspect, wherein the parameter includes at least one of noiseinformation, distortion aberration information, and peripheralillumination information. The image processing includes processing forremoving noise, processing for adding noise, processing for correctingfor distortion aberration, and processing for correcting for peripheralillumination.

Here, at least one of the processing for removing noise, processing foradding noise, processing for correcting for distortion aberration, andprocessing for correcting for peripheral illumination of not only thefirst image but also the second image is executed using at least one ofthe noise information, distortion aberration information, and peripheralillumination information on the first image that has been estimated fromthe first image. The noise information, distortion aberrationinformation, and peripheral illumination information are parameters thatdepend on shooting conditions such as the camera, shooting time of day,and shooting location. Accordingly, it is possible to obtain appropriateimages while increasing the speed of the noise removal processing, thedistortion aberration correction processing, and the peripheralillumination correction processing of a plurality of images taken undersimilar conditions.

A method of image processing according to an eighteenth aspect includesa first image processing step and a second image processing step. Thefirst image processing step is a step of calculating a parameter forimage processing based on a first image, but not based on a secondimage, and executing the image processing on the first image using theparameter. The second image processing step is a step of executing theimage processing on the second image using the parameter.

Here, the same image processing is executed on the first image and thesecond image. In addition, the parameter for use in the image processingof both of the images is calculated from only the first image of thefirst and second images. In other words, the parameter calculated fromthe first image for use in the image processing of the first image isalso used for the image processing of the second image. It should benoted that as the manner of making use of the parameter calculated fromthe first image for the second image, a manner in which the parametercalculated from the first image is used as it is for the imageprocessing of the second image, a manner in which the parametercalculated from the first image is fine-tuned before being used for theimage processing of the second image, and the like are conceivable. Thatis to say, processing for calculating a parameter for use in the imageprocessing of the second image is omitted or simplified. Accordingly,computation time taken to execute the same image processing on aplurality of images is reduced.

A method of image processing according to a nineteenth aspect is themethod according to the eighteenth aspect, wherein the parameterincludes an extent of blurring. The image processing includes processingfor enhancing an edge.

Here, the processing for enhancing edges in not only the first image butalso the second image is executed using the extent of blurring of thefirst image that has been estimated from the first image. The extent ofblurring is expressed as, for example, a point spread function (PSF).The extent of blurring is a parameter that depends on the properties oflenses of a camera when it is resulted from in defocusing. Therefore, ifthe first and the second images are images taken by the same camera, theextent of blurring estimated from the first image can be applied to thesecond image. Here, it is possible to obtain appropriate images whileincreasing the speed of edge enhancement processing of a plurality ofimages that have been taken under similar conditions, when the samecamera was used to take those images or the like.

A method of image processing according to a twentieth aspect is themethod according to the eighteenth aspect, wherein the parameterincludes at least one of noise information, distortion aberrationinformation, and peripheral illumination information. The imageprocessing includes processing for removing noise, processing for addingnoise, processing for correcting for distortion aberration, andprocessing for correcting for peripheral illumination.

Here, at least one of the processing for removing noise, processing foradding noise, processing for correcting for distortion aberration, andprocessing for correcting for peripheral illumination of not only thefirst image but also the second image is executed using at least one ofthe noise information, distortion aberration information, and peripheralillumination information on the first image that has been estimated fromthe first image. The noise information, distortion aberrationinformation, and peripheral illumination information are parameters thatdepend on shooting conditions such as the camera, shooting time of day,and shooting location. Accordingly, it is possible to obtain appropriateimages while increasing the speed of the noise removal processing, thedistortion aberration correction processing, and the peripheralillumination correction processing of a plurality of images taken undersimilar conditions.

Advantageous Effects of Invention

According to the present invention, the parameter that has beencalculated from the first image for use in the image processing of thefirst image is also used for the image processing of the second image.That is to say, processing for calculating a parameter for use in theimage processing of the second image is omitted or simplified.Accordingly, computation time taken to execute the same image processingon a plurality of images is reduced.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an image processing apparatus according toan embodiment of the present invention.

FIG. 2 shows a basic screen before image data is captured.

FIG. 3 shows the basic screen after image data has been captured.

FIG. 4 shows a confirmation window.

FIG. 5 is a flowchart of a blur correction process to be executed on aselected frame group.

FIG. 6 is a flowchart of the blur correction process to be executed oneach frame.

FIG. 7 is a flowchart of a blur correction process to be executed oneach frame according to a variation.

REFERENCE SIGNS LIST

-   -   1 Image processing apparatus (computer)    -   2 Image processing program    -   41 Image processing unit    -   41-L Lth image processing unit    -   42 Instruction receiving unit

DETAILED DESCRIPTION OF THE INVENTION

Hereinafter, an image processing apparatus, a recording medium storingan image processing program, and a method of image processing accordingto an embodiment of the present invention will be described withreference to the drawings.

1. Brief Outline of Image Processing Apparatus

An image processing apparatus 1 shown in FIG. 1 is an embodiment of animage processing apparatus according to the present invention. The imageprocessing apparatus 1 is a general-purpose personal computer. An imageprocessing program 2, which is an embodiment of an image processingprogram according to the present invention, is installed in the imageprocessing apparatus 1 from a computer-readable recording medium 60 suchas a CD-ROM, a DVD-ROM, a Blu-ray disc, a USB memory, or the like. Theimage processing program 2 is application software for assisting imageprocessing of moving images and still images. The image processingprogram 2 causes the image processing apparatus 1 to execute stepscontained in an operation that will be described later.

The image processing apparatus 1 has a display 10, an input unit 20, astorage unit 30, and a control unit 40. The display 10, the input unit20, the storage unit 30, and the control unit 40 are connected to oneanother by a bus line 5 and can communicate with one another. In thepresent embodiment, the display 10 may be a liquid crystal display. Theinput unit 20 may be composed of a mouse, a keyboard, and the like. Thestorage unit 30 may be composed of a hard disk and the like. The controlunit 40 may be composed of a CPU, a ROM, a RAM, and the like.

The display 10 displays screens that will be described later and thelike to a user. The input unit 20 receives a user operation on the imageprocessing apparatus 1.

The image processing program 2 is stored in the storage unit 30. Asoftware management region 50 is set aside in the storage unit 30. Thesoftware management region 50 is a region that is used by the imageprocessing program 2. An original image region 51 and a manipulated fileregion 52 are set aside in the software management region 50. Thefunctions of these regions 51 and 52 will be described later.

The control unit 40 reads out and executes the image processing program2 stored in the storage unit 30, thereby virtually operating as an imageprocessing unit 41 and an instruction receiving unit 42. The operationsof these units 41 and 42 will be described later.

2. Details of Configuration and Operation of Image Processing Apparatus

When the control unit 40 detects that the user has performed apredetermined operation through the input unit 20, the control unit 40activates the image processing program 2. When the image processingprogram 2 is activated, a basic screen W1 (see FIG. 2) is displayed onthe display 10. It should be noted that the control unit 40 controls thedisplay of screens, windows, buttons, and all other elements displayedon the display 10.

2-1. Image Data Capture

The basic screen W1 receives a command from the user to capture imagedata into the original image region 51. The image data captured into theoriginal image region 51 will be a target of image processing that willbe described later. The control unit 40 captures image data from a stillimage file or a moving image file into the original image region 51. Itshould be noted that in this specification, “still image file” refers toa data file in a still image format, and “moving image file” refers to adata file in a moving image format.

When capturing image data from a still image file, the user designates astill image file or a folder by operating the input unit 20. In theformer case, the control unit 40 prompts the user to input an addresspath in the storage unit 30 and a file name of that still image file. Inthe latter case, the control unit 40 prompts the user to input anaddress path in the storage unit 30 and a folder name of that folder.After that, the control unit 40 stores the designated still image fileor all the still image files within the designated folder in theoriginal image region 51 as a still image file group. It should be notedthat “group” as used herein is not limited to a plurality of elements,and the number of elements of the group may be one.

On the other hand, when capturing image data from a moving image file,the user inputs an address path in the storage unit 30 and a file nameof a moving image file by operating the input unit 20. When the controlunit 40 detects that the user has designated the moving image file, thecontrol unit 40 displays a moving image capture window (not shown) suchthat this window is laid on the basic screen W1. The moving imagecapture window receives, from the user, the selection of a timeline ofany length out of the entire timeline of the designated moving imagefile. When the control unit 40 detects that the user has selected thetimeline of any length through the input unit 20, the control unit 40generates a still image file group corresponding to that selection. Thisstill image file group corresponds one-to-one to a frame group that iscontained in a moving image of the timeline selected by the user.Afterward, the control unit 40 stores this still image file group in theoriginal image region 51.

Accordingly, in the present embodiment, the target of image processingthat will be described later is not a moving image file but a stillimage file. The still image file is captured into the original imageregion 51 on a file-by-file basis, on a folder-by-folder basis, or inthe unit of the whole or a partial timeline of the moving image file.

2-2. Reproduction of Still Image File Group

When the still image file group is captured into the original imageregion 51, the control unit 40 displays a display window W2 (see FIG. 3)such that this window is laid on the basic screen W1. The displaywindows W2 is displayed for each capturing operation for capturing astill image file group into the original image region 51.

In the display window W2, first, a single still image file (for example,a still image file corresponding to the leading frame on the timeline)contained in the still image file group that has been captured into theoriginal image region 51 is displayed. It should be noted that thecontrol unit 40 recognizes that still image files contained in the stillimage file group are arranged along the timeline, even if the stillimage file group is derived from a still image file rather than beingderived from a moving image file. The arrangement is automaticallydetermined based on the attributes (file name, date and time ofcreation, date and time of update, and the like) of a file.

As will be described later, the frame displayed in the display window W2changes in response to a user operation. The control unit 40 managesidentification information of the frame that is currently displayed inthe display window W2 in real time.

The control unit 40 can reproduce, within the display window W2, thestill image file group corresponding to that display window W2 as amoving image. As shown in FIG. 3, the control unit 40 displays a windowselection pull-down menu T1, a reproduce button T2, a frame forwardbutton T3, a frame back button T4, and a timeline bar T5 on the basicscreen W1.

Even when there is a plurality of display windows W2, only one of thosedisplay windows W2 is active. The window selection pull-down menu T1receives the selection of which display window W2 is to be made activefrom the user. Hereinafter, a still image file group corresponding to anactive display window W2 will be referred to as “active file group”.Moreover, a frame that is currently displayed in an active displaywindow W2 will be referred to as “active display frame”.

The reproduce button T2 receives a command from the user to reproducethe active file group as a moving image. When the control unit 40detects that the user has pressed the reproduce button T2 through theinput unit 20, the control unit 40 displays frames of the active filegroup in the active display window W2 in such a manner that the framesare sequentially displayed along the timeline. It should be noted thatthe reproduction begins with an active display frame at the point intime when the reproduce button T2 is pressed. Moreover, the reproducebutton T2 receives a command from the user to stop reproduction. Whenthe control unit 40 detects that the user has pressed the reproducebutton T2 through the input unit 20 during reproduction, the controlunit 40 fixes the display in the active display window W2 to an activedisplay frame at that point in time.

The frame forward button T3 and the frame back button T4 respectivelyreceive commands from the user to change the active display frame to aframe directly after and directly before that active display frame alongthe timeline of the active file group.

The timeline bar T5 graphically represents the timeline of the activefile group. The timeline bar T5 is equally divided in an extendingdirection of the bar into the same number of parts as the number offrames of the active file group. An “n”th divided region from the lefton the timeline bar T5 corresponds to an “n”th frame of the active filegroup on the timeline (“n” is a natural number).

As shown in FIG. 3, in the timeline bar T5, divided regions A1corresponding to a selected frame group and divided regions A2corresponding to a non-selected frame group are displayed differently.“Selected frame group” refers to a frame group corresponding to asection that is currently selected on the timeline of the active filegroup. “Non-selected frame group” refers to a frame group correspondingto a section that is not currently selected on the timeline of theactive file group. In the present embodiment, the region A1 is displayedin a light tone of color, and the region A2 is displayed in a dark toneof color.

The timeline bar T5 receives the selection of any section of the activefile group on the timeline from the user. The section that is selectedmay be a continuous section or may be a discontinuous section. In otherwords, the user can select any number of any frames out of all theframes of the active file group by operating the timeline bar T5 throughthe input unit 20. Specifically, the user selects a divided regioncorresponding to a frame that he/she desires to select on the timelinebar T5. It is possible to select a plurality of divided regions at thesame time. The image processing unit 41 recognizes the selected framegroup as the target of image processing that will be described later. Itshould be noted that each time the user selects a divided region on thetimeline bar T5, the active display frame is changed to a framecorresponding to the latest selected divided region.

2-3. Image Processing

Hereinafter, image processing of the selected frame group will bedescribed. The image processing unit 41 can execute a plurality of imageprocessing modules, such as noise removal, noise addition, sharpness,brightness/contrast/saturation adjustment, image resolution, rotation,addition of characters/arrows/mosaics, distortion aberration correction,peripheral illumination correction, and blur correction. The imageprocessing modules are incorporated in the image processing program 2.

The instruction receiving unit 42 receives an instruction from the userto execute image processing. The user can select any modules of theimage processing modules in any order and any number of times byoperating the basic screen W1 through the input unit 20. If necessary,the user inputs a parameter that is used during execution of an imageprocessing module at the same time as he/she selects that imageprocessing module. Each time the instruction receiving unit 42 detectsthat the user has selected an image processing module, the imageprocessing unit 41 executes that image processing module on the selectedframe group. It should be noted that executing an image processingmodule on a selected frame group means executing that image processingmodule on each frame contained in that selected frame group.

As image processing modules are executed on a frame sequentially, thatis, once, twice, thrice, and so on, that frame is sequentiallymanipulated into a first-order frame, a second-order frame, athird-order frame, and so on. A zeroth-order frame corresponds to astill image file stored in the original image region 51. An(m+1)th-order frame corresponds to a still image file after an imageprocessing module is executed once on an “m”th-order frame (“m” is aninteger of 0 or more). As image processing modules are sequentiallyexecuted, still image files corresponding to the first-order andsubsequent frames are sequentially generated. These still image filesare stored in the manipulated file region 52 as separate files.

2-3-1. Blur Correction

Details of image processing for blur correction (hereinafter referred toas “blur correction process”) will be described below. As describedabove, the instruction receiving unit 42 receives an instruction fromthe user to execute the blur correction process. The image processingunit 41 starts the blur correction process illustrated in FIG. 5 on aselected frame group in response to the user instruction to execute theblur correction process.

Specifically, when the control unit 40 detects that the user hasperformed a predetermined operation on the basic screen W1 through theinput unit 20, the control unit 40 displays a confirmation window W3(see FIG. 4) such that this window is laid on the basic screen W1.Processing of steps S22 and S23, which will be described later, includedin the blur correction process is high-load processing. Accordingly,when the blur correction process is started, any other process may beretarded. The confirmation window W3 is a window that displays a warningthat the blur correction process takes time to the user and asks theuser whether or not to still execute the blur correction process.

The confirmation window W3 displays an execution button B1 and a cancelbutton B2. The instruction receiving unit 42 receives an instructionfrom the user to execute the blur correction process through theexecution button B1. The instruction receiving unit 42 receives aninstruction from the user to cancel execution of the blur correctionprocess through the cancel button B2. When the instruction receivingunit 42 detects that the user has pressed the execution button B1through the input unit 20, the image processing unit 41 starts the blurcorrection process illustrated in FIG. 5.

Specifically, the image processing unit 41 sequentially executesprocessing of step S2 on all the frames contained in the selected framegroup. Processing of step S2 is the processing for executing a blurcorrection image processing module. In step S1 before step S2, a singleframe that is the leading frame along the timeline among frames that arecontained in the selected frame group and have not yet undergoneprocessing of step S2 is selected as a frame (hereinafter referred to as“target frame”) that is a target of processing of step S2. Therefore, inthe blur correction process illustrated in FIG. 5, processing of stepsS1 and S2 is executed K times (K is an integer of 1 or more). Krepresents the number of frames contained in a selected frame group.

When an Lth cycle of step S2 is executed, that is, when the target frameis an Lth frame, the image processing unit 41 operates as an Lth imageprocessing unit 41-L (L is a natural number). The Lth frame refers to aframe that is selected in an Lth cycle of step S1.

FIG. 6 illustrates details of processing of step S2 in FIG. 5. First, instep S21, the Lth image processing unit 41-L determines whether or notthe target frame is the first frame. If the Lth image processing unit41-L determines that the target frame is the first frame, the process isadvanced to step S22, and if not, step S22 is skipped over, and theprocess is advanced to step S23. In step S22, a parameter calculationroutine is executed, and in step S23, a blur correction routine isexecuted. The parameter calculation routine and the blur correctionroutine are invoked as subroutines during execution of the blurcorrection image processing module. Programs of the parametercalculation routine and the blur correction routine are bothincorporated in the image processing program 2.

In the parameter calculation routine of step S22, a parameter for use inthe blur correction routine of step S23 is calculated. In the presentembodiment, a point spread function (PSF) is calculated as the parameterfor the blur correction routine.

As can be seen from FIGS. 5 and 6, the blur correction routine of stepS23 is executed on each frame contained in the selected frame group,whereas the parameter calculation routine of step S22 is executed ononly the first frame of all the frames contained in the selected framegroup. That is to say, only an image processing unit 41-1 calculates thePSF, and image processing units 41-2, 41-3, . . . do not calculate aPSF. Accordingly, when the blur correction routine is repeatedlyexecuted, even though a PSF is necessary for each frame contained in theselected frame group, the PSF is calculated only once. Thus, in thepresent embodiment, during execution of the blur correction routine, thePSF for the first frame is also used as PSFs for the remaining framescontained in the selected frame group as will be described later.

In the present embodiment, the PSF is calculated in the followingmanner. In the present embodiment, the PSF is a function thatapproximates defocusing of the target frame, and the PSF is modeled as afunction below, assuming that it conforms to a Gaussian distribution.

h(x,y)=(½πσ²)exp{−(x ² +y ²)/2σ²}  (Formula 1)

Here, h(x,y) is the PSF, x,y is an orthogonal coordinate system of aspatial domain of the target frame, π is the ratio of the circumferenceof a circle to its diameter, and σ is a standard deviation. The standarddeviation σ is a parameter indicating the extent of blurring of animage. It should be noted that as can be seen from Formula 1,calculating the PSF is equivalent to calculating the standard deviationσ, which is an unknown coefficient of the model formula of the PSF.Various methods are known as the method for calculating the standarddeviation σ, and those skilled in the art can choose any method asappropriate, such as a method in which the standard deviation σ isestimated from the range of change in the luminance value of an edgeportion contained in the target frame. It should be noted that duringcalculation of the PSF in step S22, only the target frame of all theframes contained in the selected frame group is referred to, and theother frames are not referred to.

The blur correction routine of step S23 is the processing for reducingblurring contained in the target frame and enhancing edges. In step S23,the Lth image processing unit 41-L executes the blur correction routineon the target frame using the PSF (more precisely, the standarddeviation σ in the present embodiment) that has been calculated in stepS22. That is to say, the Lth image processing unit 41-L estimates thetarget frame before being blurred based on the PSF. Specifically, in thepresent embodiment, the Lth image processing unit 41-L has a frameG(u,v) that is the Fourier transform of a target frame g(x,y) passthrough an inverse filter H_(inv)(u,v) that is the inverse of a functionH(u,v) obtained by Fourier transforming h(x,y), which is the PSF, andobtains the inverse Fourier transform of a frame F(u,v) that has passedthrough the inverse filter H_(inv)(u,v), thereby generating an originalframe f(x,y) in which blurring is reduced. It should be noted that u,vis an orthogonal coordinate system of a frequency domain of the targetframe. This algorithm is based on Formula 2 below.

F(u,v)=H _(inv)(u,v)G(u,v)  (Formula 2)

In the above-described manner, when processing of step S2 is finishedwith respect to all the frames contained in the selected frame group,the blur correction process illustrated in FIG. 5 is completed.

3. Applications

As described above, irrespective of whether the image processing program2 captures image data from a still image file or a moving image file,the image processing program 2 thereafter manages the captured data as astill image file group. Therefore, the image processing program 2 caneasily meet the user's request to perform image processing as desired byhim/her on any frame in the same timeline. Moreover, the imageprocessing program 2 has a function of reproducing the still image filegroup as a moving image. Consequently, even though the image data istreated as the still image file group in the image processing program 2,the user can recognize that image data as a moving image. Therefore, theimage processing program 2 is particularly useful in instances where amoving image is analyzed, edited, and so on.

The image processing program 2 can deal with image processing of variouskinds of moving images, and can be used, for example, in order for anorganization such as the police to analyze a monitoring image from asecurity camera for investigation of a case. For example, there arecases where a suspect is recorded in a monitoring image from a securitycamera. However, usually, security cameras are seldom focused on thesubject (suspect), and important images are often unclear. The blurcorrection function of the present embodiment is particularly useful inrapidly sharpening an unclear image group which is contained in amonitoring image from a security camera and in which defocusing or thelike occurs.

4. Features

4-1

According to the above-described embodiment, the image processing unit41 calculates a point spread function (PSF), which serves as theparameter of the blur correction routine, based on the first frame ofall the frames contained in the selected frame group, and executes theblur correction routine on the first frame using the PSF. Moreover, theimage processing unit 41 also executes the blur correction routine onthe frames after the first frame contained in the selected frame groupusing the same PSF.

That is to say, a common PSF for all the frames contained in theselected frame group is calculated from only the first frame of all theframes. In other words, a PSF that has been calculated from the firstframe for the blur correction routine to be executed on the first frameis used as it is as PSFs for the blur correction routine to be executedon the second and subsequent frames. This means that processing forcalculating the PSFs from the second and subsequent frames for the blurcorrection routine to be executed on the second and subsequent frames isomitted. Accordingly, computation time taken to execute the same blurcorrection routine on a plurality of frames is reduced.

4-2

The extent of blurring of an image may depend on the properties oflenses of a camera, such as defocusing. Therefore, the extent ofblurring estimated from a certain frame can be applied to another frameif the frames have been taken by the same camera. According to theabove-described embodiment, the extent of blurring of the first framecontained in the selected frame group that has been estimated from thefirst frame is used to execute the blur correction routine on not onlythe first frame but also the second and subsequent frames. Thus, byselecting a plurality of frames taken under similar conditions, forexample, when the same camera was used to take the frames, as theselected frame group, the user can obtain appropriate images in whichblurring are reduced while increasing the speed of the blur correctionprocess executed on those frames.

4-3

In many cases, it may be appropriate to execute the same imageprocessing using the same parameter on a plurality of frames that is atleast a part of frames belonging to the same timeline. This is becausethose frames are highly likely to have been taken under similarconditions. According to the above-described embodiment, it is possibleto execute the blur correction routine using the same PSF on theselected frame group containing a plurality of frames on the sametimeline. Accordingly, it is possible to obtain appropriate images inwhich blurring are reduced while increasing the speed of the blurcorrection process.

5. Variations

Although an embodiment of the present invention has been describedabove, the present invention is not limited to the above embodiment, andvarious changes can be made thereto without departing from the gist ofthe present invention. For example, the following changes can be made.

5-1

In step S22, the parameter was calculated from the leading frame on thetimeline of all the frames contained in the selected frame group;however, the parameter may also be calculated from a frame other thanthe leading frame. For example, the frame based on which the parameteris calculated may be freely designated by the user, or may beautomatically specified in accordance with a predetermined algorithm inwhich a frame other than the leading frame may be selected.

5-2

In step S22, the parameter was calculated from only a single frame ofall the frames contained in the selected frame group; however, theparameter may also be calculated from two or more frames. For example,it is also possible to calculate the parameter from only the first andsecond frames on the timeline of all the frames contained in theselected frame group and use this parameter for the blur correctionroutine of not only the first and second frames but also the third andsubsequent frames.

5-3

In the above-described embodiment, processing of step S23 executed onall the frames contained in the selected frame group was sequentialprocessing; however, processing of step S23 may also be parallelprocessing. For example, it is also possible that after the parameter iscalculated in step S22, the image processing units 41-1, 41-2, . . .respectively execute processing of step S23 on different frames inparallel.

5-4

The above-described embodiment was configured so that the selected framegroup, which is the target of execution of an image processing module,can only be selected from frames on the same timeline; however, it isalso possible that the selected frame group can be selected from frameson different timelines.

Generally, there are cases where it is appropriate to execute the sameimage processing using the same parameter on a plurality of framesbelonging to different timelines. For example, this holds in the casewhere those frames have been taken by the same camera installed at thesame location. According to this variation, it is possible to executethe blur correction routine using the same PSF on a selected frame groupcontaining a plurality of frames belonging to different timelines.Therefore, it is possible to obtain appropriate images in which blurringare reduced while increasing the speed of the blur correction process.

5-5

The above-described embodiment was configured so that the standarddeviation σ of the PSF is calculated as the parameter for use in theblur correction routine to be executed on a plurality of frames;however, another parameter indicating the extent of blurring may also becalculated. For example, it is also possible to model the PSF as afunction other than a Gaussian function and calculate another parametercontained in that model. Alternatively, an indicator other than the PSFmay be used as the parameter indicating the extent of blurring.

5-6

The above-described embodiment was configured so that the blurcorrection routine is executed as image processing of step S23 in whichthe parameter calculated in step S22 is shared among a plurality offrames; however, other image processing may also be executed. It shouldbe noted that naturally, if image processing of step S23 changes, theparameter that should be calculated in step S22 also changes.

For example, it is also possible that in step S23, processing forremoving noise or processing for adding noise is executed, and in stepS22, noise information (parameter) of a noise removal filter, a noiseaddition filter, or the like is calculated from a specific frame of theselected frame group.

Alternatively, it is possible that in step S23, processing forcorrecting for distortion aberration is executed, and in step S22,information (parameter) indicating the extent of distortion aberrationsuch as a distortion factor is calculated from a specific frame of theselected frame group.

Alternatively, it is possible that in step S23, processing forcorrecting for peripheral illumination is executed, and in step S22,information (parameter) that has an influence on the extent ofperipheral illumination such as the angle of view is calculated from aspecific frame of the selected frame group.

5-7

The above-described embodiment was configured so that the parametercalculated from the first frame of all the frames contained in theselected frame group is used as it is as not only the parameter for thefirst frame but also parameters for the remaining frames. However, themethod for making use of the parameter for the first frame for imageprocessing of the remaining frames is not limited to this. For example,during image processing of the remaining frames, the parametercalculated from the first frame may be fine-tuned using information onthe remaining frames so as to become more appropriate as the parametersfor the remaining frames. According to this variation, processing forcalculating the parameters for the remaining frames is simplified, andcomputation time taken to execute the same image processing on all theframes is reduced. When this variation is applied to step S2 of theabove-described embodiment, for example, processing of step S2illustrated in FIG. 6 may be changed to processing as illustrated inFIG. 7. That is to say, the processing may be changed so that withrespect to frames other than the first frame, instead of step S22, inwhich the parameter is calculated from the beginning, step S24 isexecuted in which the parameter that has already been calculated in stepS22 is fine-tuned.

5-8

The above-described embodiment was configured so that any image datacaptured into the original image region 51 is stored as a still imagefile. However, it is also possible to store the image data captured intothe original image region 51 as a moving image file. Whether the imagedata in the original image region 51 is stored in a still image formator a moving image format may be user-selectable or may be automaticallydetermined according to the format or the like of the source image data.

Furthermore, after that, if the moving image file is subjected to imageprocessing, the image data after this image processing may also bestored in the format of the moving image file.

What is claimed is:
 1. An image processing apparatus comprising: a firstimage processing unit configured to calculate a parameter for imageprocessing based on a first image, but not based on a second image, andexecute the image processing on the first image using the parameter; anda second image processing unit configured to execute the imageprocessing on the second image using the parameter.
 2. The imageprocessing apparatus according to claim 1, wherein the first image andthe second image are images on the same timeline.
 3. The imageprocessing apparatus according to claim 1, wherein the first image andthe second image are images on different timelines.
 4. The imageprocessing apparatus according to claim 1, wherein the parameterincludes an extent of blurring, and the image processing includesprocessing for enhancing an edge.
 5. The image processing apparatusaccording to claim 2, wherein the parameter includes an extent ofblurring, and the image processing includes processing for enhancing anedge.
 6. The image processing apparatus according to claim 3, whereinthe parameter includes an extent of blurring, and the image processingincludes processing for enhancing an edge.
 7. The image processingapparatus according to claim 1, wherein the parameter includes at leastone of noise information, distortion aberration information, andperipheral illumination information, and the image processing includesat least one of processing for removing noise, processing for addingnoise, processing for correcting for distortion aberration, andprocessing for correcting for peripheral illumination.
 8. The imageprocessing apparatus according to claim 2, wherein the parameterincludes at least one of noise information, distortion aberrationinformation, and peripheral illumination information, and the imageprocessing includes at least one of processing for removing noise,processing for adding noise, processing for correcting for distortionaberration, and processing for correcting for peripheral illumination.9. The image processing apparatus according to claim 3, wherein theparameter includes at least one of noise information, distortionaberration information, and peripheral illumination information, and theimage processing includes at least one of processing for removing noise,processing for adding noise, processing for correcting for distortionaberration, and processing for correcting for peripheral illumination.10. The image processing apparatus according to claim 1, furthercomprising: an instruction receiving unit configured to receive aninstruction to execute the image processing on the first image and thesecond image, wherein the first image processing unit is configured tocalculate the parameter and execute the image processing on the firstimage in response to the instruction, and the second image processingunit is configured to execute the image processing on the second imagewhen the parameter has been calculated.
 11. The image processingapparatus according to claim 2, further comprising: an instructionreceiving unit configured to receive an instruction to execute the imageprocessing on the first image and the second image, wherein the firstimage processing unit is configured to calculate the parameter andexecute the image processing on the first image in response to theinstruction, and the second image processing unit is configured toexecute the image processing on the second image when the parameter hasbeen calculated.
 12. The image processing apparatus according to claim3, further comprising: an instruction receiving unit configured toreceive an instruction to execute the image processing on the firstimage and the second image, wherein the first image processing unit isconfigured to calculate the parameter and execute the image processingon the first image in response to the instruction, and the second imageprocessing unit is configured to execute the image processing on thesecond image when the parameter has been calculated.
 13. The imageprocessing apparatus according to claim 4, further comprising: aninstruction receiving unit configured to receive an instruction toexecute the image processing on the first image and the second image,wherein the first image processing unit is configured to calculate theparameter and execute the image processing on the first image inresponse to the instruction, and the second image processing unit isconfigured to execute the image processing on the second image when theparameter has been calculated.
 14. The image processing apparatusaccording to claim 7, further comprising: an instruction receiving unitconfigured to receive an instruction to execute the image processing onthe first image and the second image, wherein the first image processingunit is configured to calculate the parameter and execute the imageprocessing on the first image in response to the instruction, and thesecond image processing unit is configured to execute the imageprocessing on the second image when the parameter has been calculated.15. A non-transitory computer-readable recording medium storing an imageprocessing program configured to cause a computer to execute steps of:calculating a parameter for image processing based on a first image, butnot based on a second image, and executing the image processing on thefirst image using the parameter; and executing the image processing onthe second image using the parameter.
 16. The non-transitorycomputer-readable recording medium according to claim 15, wherein theparameter includes an extent of blurring, and the image processingincludes processing for enhancing an edge.
 17. The non-transitorycomputer-readable recording medium according to claim 15, wherein theparameter includes at least one of noise information, distortionaberration information, and peripheral illumination information, and theimage processing includes at least one of processing for removing noise,processing for adding noise, processing for correcting for distortionaberration, and processing for correcting for peripheral illumination.18. A method of image processing, comprising steps of: calculating aparameter for image processing based on a first image, but not based ona second image, and executing the image processing on the first imageusing the parameter; and executing the image processing on the secondimage using the parameter.
 19. The method of image processing accordingto claim 18, wherein the parameter includes an extent of blurring, andthe image processing includes processing for enhancing an edge.
 20. Themethod of image processing according to claim 18, wherein the parameterincludes at least one of noise information, distortion aberrationinformation, and peripheral illumination information, and the imageprocessing includes at least one of processing for removing noise,processing for adding noise, processing for correcting for distortionaberration, and processing for correcting for peripheral illumination.